feat(boards): share boards by role and keep project settings per member - #9797

I reviewed this at 83d4a29 by trying to break each area: board authorization, migrations and SQL, the per-member workspace split, and the gallery client. The PR's own tests pass. I ran the migrations on 6.13- and alpha.1-shaped SQLite databases. I had no MySQL/MariaDB server, so I checked the server path by reading the code and compiling its DDL offline. I found two blockers. Both lose or corrupt members' data.

Blockers

1. The lazy split from schema 3 to 4 can permanently wipe a member's workspace

durableSyncedPersistence.ts:1510-1557 (pushHalves) splits a whole (schema-3) project in two separate requests. It first sends the content PUT with minimum_document_schema_version: 4, then sends PUT /projects/{id}/workspace. On the server, ProjectRecordService.update (project_records_default.py:298) replaces projects.data with the content alone. Between the two requests, the server no longer holds the member's layout, generation settings, prompt history, selection or staging.

recordRequiresDocumentUpgrade (:377-380) makes this split happen on the first save after opening, even when nothing changed. No user action is needed to trigger it.

Triggering sequence:

1. Open a project saved before this PR.
2. The content PUT commits.
3. Before the workspace PUT lands, the user signs out. endLifetime aborts owner.signal, so assertOwner() throws before the workspace request is sent. accountOwnedWorkbenchDatabase.ts:51-59 then deletes the account's IndexedDB, local draft included. A tab closing, or a private window with no draft store plus a failed request, ends the same way.
4. Every later load joins the content with createDefaultProjectWorkspace(). The member's settings are gone.

Before this PR every save was one whole-document transaction, so a save that only half landed could not destroy data already on the server. In the same window, media_references.replace(owner_kind="project") (:307) also drops the references to staged canvas candidates. Those candidates are intermediates, so cleanup can delete them while a surviving draft still names them.

Suggested fix: do the upgrade in one server transaction (content plus the first workspace row in one request), or keep the whole document until the workspace row exists.

2. Losing access to a board is now treated as the board being deleted

_require (routers/_access.py:22-37) now answers 404 for a board the caller can't read. On the base it answered 403, and the old get_board comment explained why: "clients take it as the board being gone, and clear what names it". The client side was not changed:

• WorkbenchRuntime.tsx:46-75 checks every board a project names (selected board, auto-add board, queue items) through galleryBoardExists (gallery/data/backend.ts:1071-1087).
• galleryBoardExists maps a 404 to "doesn't exist".
• That runs reconcileDeletedGalleryBoard (workbenchState.ts:3051-3104).

Triggering sequence:

1. A owns board X with general access Everyone can contribute.
2. B selects X and generates into it.
3. A sets X to Restricted.
4. B reopens the project. In B's saved workspace:
   • selectedBoardId becomes 'none', so later results go to Uncategorized, not to the project Inbox the PR describes.
   • The auto-add board is reset.
   • Every locally known item on X, including B's own images that are still on X on the server, gets boardId: 'none'.

None of this is undone if A grants access again. A reversible permission change triggers an irreversible cleanup. Either the server needs a status that separates "gone" from "denied" (as #9394 did for media), or the client must stop treating this 404 as deletion.

Non-blocking

• Admin owners still permanently delete other people's media. delete_board (boards.py:174) sets cascade_user_id = None for any admin. So an admin deleting their own shared board with include_images=true hard-deletes other members' uploads instead of returning them to their owners' Uncategorized. admin_override reports false in that case (board_access_common.py:109), so the UI can't warn. test_delete_board_with_include_images_admin_skips_user_filter asserts this behaviour.
• project_id leaks through item DTOs. _describe hides project_id on a BoardDTO the caller doesn't own. ImageDTO.project_id / VideoDTO.project_id are returned unredacted to anyone who can view a project member board shared with them, through GET /images/i/{name}, the list endpoints and by-names. Combined with the 409 on project create for an existing id, this confirms the project exists.
• A revoked or view-only destination is caught only at save time, and nothing refreshes afterwards.
  • There is no access-change event, and refetchOnWindowFocus is false. A cached add_media stays true after a downgrade.
  • With no listing cached at all (gallery widget closed for more than 5 minutes, or straight after startup), canAddToCachedGalleryBoard returns listings.length === 0 (queryCache.ts:480). That lets a view-only board through.
  • Either way the run fails in ImagesInterface.save after the whole generation. Boards are re-fetched only after a successful result (features/queue/runtime.ts:1054,1125,1244), so every later run fails the same way.
  • Failed uploads behave the same: useGalleryUploadAction.ts:160-168 returns without invalidateGallery.
• Explicit board ids in workflow board fields are never checked. resolveBoardSentinel (buildGraph.ts:71-81) redirects only auto and empty values. BoardInput's option list is cached for the whole session (boardOptionsRequest), so it doesn't follow grants or revocations.
• A board created a moment ago can be overwritten by an older listing. A boards fetch started before createBoard can land between the optimistic patch and scheduleGalleryInvalidation's board invalidation (queryCache.ts:690-717). Results then go to the Inbox, and no Auto badge is shown.
• Whether results reach an archived board depends on cache timing. Whether results go to a hidden archived board depends on whether an include-archived listing is still cached (queryCache.ts:455-480).
• The workspace save ignores the canvas schema floor. save_workspace checks only the document floor, and ProjectWorkspaceSaveRequest has no max_canvas_schema_version. Yet the workspace holds canvas-schema state (stagingArea, selectedLayerId).
• The new migrations never run against MySQL/MariaDB in CI. All three test_migration_2026_10_10_* files call Database.open_sqlite directly, so they don't get the uses_database mark. The server branches therefore aren't exercised with data: the duplicate re-key, the DROP PRIMARY KEY, ADD PRIMARY KEY swap, and the shared_boards copy. test_portable_migrations runs that DDL on empty tables only.
• A share on a board with a NULL owner is dropped. migration_2026_10_10_board_roles.py:68 filters on _shared_boards.c.user_id != _boards.c.user_id, which is NULL when boards.user_id is NULL. or_(_boards.c.user_id.is_(None), …) fixes it. Only hand-edited databases should hit this.
• Re-keyed project documents are re-serialized differently. They go through plain json.dumps (migration_2026_10_10_global_project_ids.py:79,126). The app uses ensure_ascii=False with compact separators (project_records_default.py:42). A document near the size limit can come out over it.
• Browser state for a re-keyed id has no migration path. An open tab's unsynced edits, or an IndexedDB draft, under the old id comes back as a "project deleted" conflict. One line in the release notes is probably enough.
• Identity is resolved differently at queue time. workflow_call_batch._assert_user_can_access_board (:311-332) ignores single-user mode and reads is_admin from the demoted system user. BoardsInterface._viewer refuses a deactivated user but not a deleted one. ImagesInterface and VideosInterface refuse both.
• Single-account board folding is applied in one place only. asSingleAccountBoards runs only in useGalleryBoards. The image context menu (ImageContextMenu.tsx:1073) and the workflow BoardInput show raw DTOs, so the same board appears as the user's own in the panel and under "Other users' boards" there.
• The mock backend has drifted from the server. mock-backend.mjs:1094,1108,1762 still refuses to claim a board whose board_visibility !== 'private', and its seeded boards have no board_visibility.
• A workflow id of __proto__ loses its lastRun. projectHalves.ts:83-88 writes into a plain object literal. Only a crafted file can do this, and only the run preview is lost.

What held up

• Board authorization:
  • Every board, item, project and queue-origin route I traced goes through board_access.
  • Moves need AddMedia on the destination plus ownership of the item.
  • Removal is decided against the item's current board.
  • All delete paths use ItemAction.Delete.
  • item_action_for_changes fails closed.
  • The board_visibility alias can't be used to escalate, and sending both fields gets 422.
  • Revocation after enqueue wins at save and read time.
  • Socket audiences don't reach outsiders for a Restricted board.
• Migrations:
  • The visibility mapping holds on 6.13- and alpha.1-shaped data.
  • Re-keying rewrites every server-side reference I could find: data, boards, images, queue project_id/origin, media references and client state.
  • SQLite rolls back cleanly on a mid-migration failure, and each crash point on a server converges on rerun.
• Workspace split:
  • Workspaces are isolated per caller.
  • Old clients are refused on every write path, including those that send no version.
  • Concurrent saves of the two halves reconcile correctly.
  • The .invk v2/v3 import edge cases are handled.

Second round, at 32880d7 (fixes 492c1be, 21b03e1, 06af3ff). Thanks: both blockers from the last review are fixed, and so are most of the non-blockers. Two small backend issues are left before this can merge. One was missed by the fix; the other was introduced by it. I've also listed some follow-ups that don't need to block.

Previous blockers: fixed

• Split atomicity. update() now writes the content, the board rename, the project media references, the member's workspace and the project_workspace references in one transaction. I forced a workspace conflict on a split and the revision, floor (3) and references were left exactly as before. Staged media moves from project to project_workspace in the same commit. The split_needs_workspace guard refuses every bypass I tried, and none of this client's paths can trigger it. Lost answers, reloads, sign-out mid-request and unload-journal replay all go through the combined request and reconcile correctly.
• Revoked board treated as deleted. _require answers 403 for a board that exists but can't be read, and 404 only on BoardRecordNotFoundException. The probe is a raw column read, so a corrupt row gives a 500, never a "gone". galleryBoardExists returns true on 403, so no reconcile runs. Results go to the Inbox, the saved state keeps the board, and a regrant restores both the selection and the destination.

Also fixed:

• An admin who owns a board no longer deletes other members' media.
• The workspace save checks the canvas floor.
• Explicit board ids in workflow fields are checked, and the field's board options follow the shared listing.
• queue_viewer handles identity consistently.
• The new migrations run on the MySQL and MariaDB jobs (860 passed at this head).
• The NULL-owner share copy is fixed.
• Re-keyed documents are serialized the way the app serializes them.
• The mock backend matches the server.
• The __proto__ lastRuns case is gone.

Changes requested

1. Video recall routes still return another user's project_id

_place_gallery_video (invokeai/app/api/routers/video_recall.py:578-590) calls assert_video_read_access, then videos.get_dto, and returns the DTO without redact_project_ids. This is the only router that returns a media DTO without redacting it. I checked every get_dto call in routers/.

Trigger:

1. user1 has a video in one of their projects, on a board with Everyone can view.
2. user2 calls GET /api/v1/videos/i/{name} and gets project_id: null.
3. user2 calls POST /api/v1/recall/video/default/initial-video?video_name={name}. The response's video.project_id is user1's project id. /reference-video and /conditioning-video do the same.

This contradicts the API guide ("everyone else reads null"). Combined with the 409 on project create, it confirms the project exists. Fix: redact_project_ids([video], current_user) in _place_gallery_video, plus a test next to the existing redaction tests. The upload variants return the caller's own media and are fine.

2. The project-id migration can now fail on a client-state value, and the app won't start

migration_2026_10_10_global_project_ids.py:50 now serializes with ensure_ascii=False. In the client-state rewrite (:121-132), json.loads turns an escaped lone surrogate (e.g. "\ud83d") into a lone surrogate character. JSON.stringify writes exactly that for a string cut mid-emoji. _dumps then writes the character out raw. The UTF-8 encode fails at conn.execute, which is outside the try/except ValueError:

MigrationError: 'utf-8' codec can't encode character '\ud83d' ... surrogates not allowed

Trigger: two accounts hold the same project id (so _rekey runs), and the re-keyed account has a client_state value that contains the old id and such an escape. I reproduced it with the PR's own before fixture. With the previous json.dumps (ASCII escapes), the same input passes.

Project documents aren't affected, because _serialize_project_document already refuses lone surrogates on write. Only client_state stores the client's raw string. Duplicate ids are rare, but when this happens startup is blocked. Fix: encode inside the try (rewritten_value.encode("utf-8")) and skip the row on failure, or keep ensure_ascii=True for client state, which has no size limit to protect. A test with a surrogate escape would pin it.

Recommended follow-ups (non-blocking)

1. A failed board-list refetch sends results to the Inbox without telling the user.
   • Where: canAddToCachedGalleryBoard (features/gallery/data/queryCache.ts:464-484) filters on query.state.status === 'success'.
   • Why it happens: when a refetch fails, TanStack Query keeps the data but sets status: 'error', so the only listing drops out and the function returns false. This contradicts the comment in ensureGalleryBoardDestinations ("the cached listings, if any, decide").
   • Trigger: with show archived on, the panel's listing is also the ensured one. One of its five requests (/boards or an Uncategorized count) returns 500.
   • Effect: every submit then goes to the Inbox. The panel still shows the chosen board and no redirect badge.
   • Suggestion: accept listings whose data is defined whatever their status. Add a test that seeds the include-archived key in an error state with data.
2. Create paths still store content and the creator's workspace in two requests.
   • Where: createSaveDraftAsNew (durableSyncedPersistence.ts:146-167), the first save of a new project (the !base branch of pushHalves), and import/duplicate via saveTransferredWorkspace.
   • Effect: imports and duplicates can be redone from their source. A conflict copy ("save mine as a copy") and a new project's first save have no other copy. Signing out between the two requests wipes IndexedDB and loses the draft workspace for good.
   • This is the window this round closed for whole projects. Suggestion: let POST /projects take an optional workspace and write it in the same transaction.
3. Some destinations still skip the ensure step or don't refresh after a refusal.
   • Video reference uploads read the upload board synchronously from the observed listing (app/VideoUiAdapter.tsx:100-105, VideoReferenceListField.tsx:562-582). Nothing refreshes that listing except invalidation, and touchGalleryImages runs only on success, so a revoked auto-add board 403s on every retry.
   • A refused canvas save to gallery (useCanvasGallerySave.ts:63-70) doesn't invalidate, and the next save within 60 s reuses the fresh listing.
   • queue/runtime.ts:440-450 refreshes after a failed run only for destination === 'gallery', so a canvas-bound workflow that names a board isn't covered.
   • useGalleryUploadAction.ts:99 still trusts an unlisted id "as given".
   • Suggestion: route these through ensureGalleryBoardDestinations and invalidate on any board refusal.
4. Cost of the pre-submit listing.
   • Waiting at submit: after 60 s without an invalidation, every submit awaits a full listing (five requests) before dispatch (invocationSubmit.ts:205-208, 283-290). The preparation lease silently drops extra Invoke clicks in that window.
   • Extra request per completion: the always-mounted include-archived useQuery in VideoUiAdapter adds a second listing to every invalidateGallery.
   • Not measured: the browser baseline counts only initial asset requests, so neither cost is recorded.
   • Suggestion: measure it on the release gate. Consider resolving the destination optimistically from a cached listing and refreshing in the background, or queueing clicks that arrive during preparation instead of dropping them.
5. Small items.
   • PasteMediaDialog.tsx:116-125 passes the unfolded listing, so a single-account install labels legacy boards by their stored names.
   • useGalleryActions.ts:33,72 tells "lost" from "stale" by matching the server's detail string. A structured code in the 403 body would be sturdier.
   • PUT /projects/{id}/workspace accepts a workspace beside a whole (floor-3) project, which update() would refuse as workspace_needs_split_document. Harmless with this client, but the two routes should agree.
   • Two small inconsistencies remain in the image/video interfaces and workflow_call_batch:
     • ImagesInterface/VideosInterface still inline the deleted/deactivated rule instead of using queue_viewer, although the docstring says everything resolves through it.
     • users.get raising in workflow_call_batch now escapes as a raw exception, not UnsupportedWorkflowNodeError. It still fails closed.

Checks

• SQLite: tests/app/services/project_records, tests/app/routers/test_projects.py, the multiuser board/video/authorization suites, board_access, test_workflow_call_batch, the invocation-context board tests and the whole sqlite_migrator directory: 152 + 558 passed.
• MySQL and MariaDB: CI's jobs passed at this head (860 passed, 2 skipped). I had no server to run them locally.
• Frontend: findings come from reading the code, plus a probe against the installed TanStack Query 5.102.8 confirming that data is kept on a failed refetch.

Third round, at a818f71 (fix e24b1ed). Thanks. Both remaining blockers are fixed, and I found nothing new that blocks. Approving.

Previous blockers: fixed

• Video recall leaked project_id. _place_gallery_video now calls redact_project_ids before it returns or emits the DTO, so /initial-video, /reference-video and /conditioning-video now match the videos API. The new test fails when the redaction is reverted.
• Migration crash on a lone surrogate. Client state is now rewritten with ASCII escapes (_dumps_client_state). The output is pure ASCII, so the encode at conn.execute can't fail. Project documents keep the compact form, which is safe because their writes already refuse lone surrogates. The re-key test now includes a surrogate escape and fails when the old serializer is put back.

Also fixed from the follow-ups

• canAddToCachedGalleryBoard now counts a listing that holds data even when its refetch failed. This has a test.
• PasteMediaDialog folds boards the same way the gallery does.
• PUT /projects/{id}/workspace now refuses a whole (floor < 4) project with workspace_needs_split_document, as update() does. I checked that this client can't hit that refusal: whole creates send no workspace, and whole updates return before the workspace save.
• ImagesInterface/VideosInterface now resolve the queue user through _viewer/queue_viewer.

Possible follow-up issues (non-blocking)

1. Create paths still store content and the creator's workspace in two requests. This affects createSaveDraftAsNew (conflict copy), the first save of a new project (the !base branch of pushHalves), and import/duplicate via saveTransferredWorkspace. If the user signs out between the two requests, the conflict copy's or new project's draft workspace is lost for good. Imports and duplicates can be redone from their source. Suggestion: let POST /projects take an optional workspace and write it in the same transaction.
2. Some board destinations skip the ensure step or don't refresh after a refusal.
   • Video reference uploads (VideoUiAdapter.tsx, VideoReferenceListField.tsx) read the board from the observed listing, so a revoked auto-add board answers 403 on every retry.
   • A refused canvas save to gallery (useCanvasGallerySave.ts) doesn't invalidate the listing.
   • queue/runtime.ts refreshes after a failed run only for destination === 'gallery'.
   • useGalleryUploadAction.ts trusts an unlisted id as given.
   • Suggestion: route these through ensureGalleryBoardDestinations and invalidate on any board refusal.
3. Cost of the pre-submit listing. After 60 s without an invalidation, each submit waits for a full listing (five requests), and the preparation lease drops extra Invoke clicks during that wait. The always-mounted include-archived query in VideoUiAdapter adds a second listing on every invalidateGallery. Neither cost is measured. Suggestion: measure it on the release gate, and consider resolving the destination optimistically with a background refresh, or queueing the dropped clicks.
4. Small items.
   • useGalleryActions.ts tells "lost" from "stale" by matching the server's detail string. A structured code in the 403 body would be sturdier.
   • users.get raising in workflow_call_batch escapes as a raw exception rather than UnsupportedWorkflowNodeError. It still fails closed.

Checks

• SQLite: video recall, projects, boards/projects, project_records, the whole sqlite_migrator directory, and the invocation-context image/video tests: 478 passed, 6 skipped.
• Mutation check: reverting either blocker fix fails its new test.
• CI at this head: 20 passed, 1 skipped, including the MySQL and MariaDB jobs.
• Frontend changes were checked by reading the code and by CI only.